Integrated segmentation and depth ordering of motion layers in image sequences

We describe a method to segment and depth order motion layers simultaneously in an image sequence. Previous approaches have tended to ignore the depth ordering issue or treat it as a post-processing operation. We argue here that motion estimation and segmentation are crucially dependent on depth order and hence that the latter should form an integral part of any layering scheme. Using an explicit model of boundary ownership allowing simultaneous assignment of motions to regions and extraction of depth order, the method fuses colour region segmentations with motion estimates obtained via block correlation. The motion estimates are then updated using a depthdependent partial correlation. Experiments show the approach is effective.